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Abstract 

The concept of visual restoration via retinal prosthesis arguably started in 1992 with the discovery that some of the 
retinal cells were still intact in those with the retinitis pigmentosa disease. Two decades later, the first commercially 
available devices have the capability to allow users to identify basic shapes. Such devices are still very far from 
returning vision beyond the legal blindness. Thus, there is considerable continued development of electrode materials, 
and structures and electronic control mechanisms to increase both resolution and contrast. In parallel, the field of 
optogenetics — the genetic photosensitization of neural tissue holds particular promise for new approaches. Given that 
the eye is transparent, photosensitizing remaining neural layers of the eye and illuminating from the outside could 
prove to be less invasive, cheaper, and more effective than present approaches. As we move toward human trials in the 
coming years, this review explores the core technological and biological challenges related to the gene therapy and the 
high radiance optical stimulation requirement. 
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Introduction 

Treatments of blindness include pharmaceuticals, gene therapy, 
stem cells, and visual assistive devices. Pharmaceutical treatment 
strategies such as anti-vascular endothelial growth factor (VEGF) 
drugs for wet-age-related macular degeneration (AMD) are becoming 
increasingly available. Such treatments are also becoming increas- 
ingly effective in treating glaucoma, for example, prostaglandin 
analogs (Tomic et al., 2013), beta-adrenergic receptor antago- 
nists (Himber et al., 1987), alpha2-adrenergic agonists (Wheeler & 
Woldemussie, 2001), alpha agonists (Arthur & Cantor, 2011), 
carbonic anhydrase inhibitors (Gugleta, 2010), and physostig- 
mine (Gburkowa & Pojda, 1965). Nevertheless, such approaches 
cannot treat conditions where the underlying cause is genetic and 
cannot restore lost vision. 

For retinal degenerations caused by mutations in a single 
gene, genetic therapies have shown promise in preclinical studies 
and early stage clinical trials (for review, see e.g., Cepko, 2012; 
Boye et al., 2013; Sahel & Roska, 2013 and the article on gene 
therapies in this issue). However, this only works in cases where 
the underlying gene is known and retinal dystrophies can arise 
due to mutations in hundreds of different genes (Daiger et al., 
2013). It the long term, it may even be possible to regrow an 



Address correspondence to: Dr. Patrick Degenaar, School of EEE, 
Newcastle University, Newcastle upon Tyne NE7 7YL, UK. E-mail: 
patrick.degenaar@newcastle.ac.uk 



entire new eye utilizing stem cell approaches as demonstrated 
in principle by Eiraku and Sasai (2012). However, these may be 
limited to certain conditions, and will take some time to become 
clinically available. As such, it is very important that we also 
explore prosthetic restorative strategies to our most important sen- 
sory system. 

Bionic solutions to blindness have been considered for almost a 
century, since Forster (1929) discovered electrical stimulation 
of the visual cortex could elicit the perception of phosphenes. 
Pioneering work on visual cortical prosthesis was carried out 
in the 1960s by Brindley and Lewin (1968) and extended by 
Dobelle et al. (1974). However, the discovery of inner retinal 
preservation in the 1990s (Stone et al., 1992; Santos et al., 
1997), combined with the relative accessibility of the eye, led to 
a shift in research emphasis to retinal prostheses. In addition to 
this article, readers may want to consider the reviews on elec- 
tronic retinal prosthesis by Shepherd et al. (2013) and Picaud 
and Sahel (2014). 

Retinal prostheses are only appropriate for photoreceptor 
degenerations where there is still a functional neural link to the 
visual cortex. Current generations of prostheses have returned 
limited vision and have thus only been implanted in those with late 
stage retinitis pigmentosa (RP) (prevalence 1:3500), who have bare 
light perception (Ahuja & Behrend, 2013). In the longer term, 
other retinopathies such as AMD, are also of interest to the pros- 
thetic community as these are much larger communities (Weiland 
et al., 201 1). These can be classed into nonvascular conditions such 
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as Stargardt (Liutkeviciene et al., 2012) and dry-AMD (van Lookeren 
Campagne et al., 2014), and vascular conditions such as wet-AMD 
(de Amorim Garcia Filho et al., 2012) and diabetic retinopathies 
(Simo & Hernandez, 2014). The latter can now be treated with anti- 
VEGF pharmaceutical agents (de Amorim Garcia Filho et al., 2012; 
Holz et al., 2014). The former, while untreatable, rarely leads to 
a total loss in vision. Thus, in the majority of cases, patients may 
be legally blind, but still have navigation function and can use 
adaptive technologies to help with other activities of daily living. 
Any prosthetic treatment for these conditions would thus have 
to restore highly functional vision without damaging what cur- 
rently remains. As such, RP currently remains the priority target 
for retinal prosthesis. 

Where the retinal ganglion cells (RGCs) are destroyed, as in optic 
neuropathies such as glaucoma, it is no longer possibly to commu- 
nicate from the eye and a deeper visual prosthesis is required. This 
may be in the lateral geniculate nucleus (Pezaris & Reid, 2007) or 
in area VI of the visual cortex (Normann et al., 2009). 



Optogenetic approaches to neuron stimulation 

Classically, the electrical based neurological therapies date back 
almost two millennia, as Scribonius Largus used electrical shocks 
from torpedo fish to provide pain relief (Nutton, 1995). Many 
centuries later, Galvani was the first to demonstrate electrical nerve 
control in 1790s. But with the advent of precise electronics in the 
20th century, clinical neuroprostheses became possible. The first 
was the heart pacemaker in 1950, followed by commercial avail- 
ability of cochlear implants in the 1980s and deep brain pace- 
makers in the 1990s. These devices utilize electrodes which impart 
electrical charge into medium near nerve cells to open up electri- 
cally gated ion channels on neuron cell membranes — and thus 
enable information transfer or pacemaking. 

Other methods of nerve control have been explored. Fork first 
demonstrated optical stimulation of the sciatic nerve of a rat with 
high power infrared lasers in 1971 (Fork, 1971). This effect could 
be the result of thermal activation of transient receptor potential 
(TRP) channels (Albert et al., 2012) or perhaps simply temperature 
increases changing the intermembrane diffusion relations (Li et al., 
2013). Another approach is the physical release (Finlayson & Iezzi, 
2010) or uncaging (Shoham et al., 2005) of neurotransmitters to 
create chemical or optochemical prosthesis. 

In all of the above cases, there is no modification of the nervous 
system, only the introduction of some form of optical, chemical, or 
electrical stimulus. Optogenetics, in contrast, involves the genetic 
modification of nerve cells to become sensitive to certain wave- 
lengths of light. There are four different methods for doing this, 
each with their own characteristics: 

1 . Light sensitive ion channels 

2. Light sensitive charge pumps 

3. Light sensitive signal transduction pathways 

4. Genetically modified receptors to which an optically activated 
ligand is chemically attached. 

Light sensitive proteins have been studied for many years. For 
example, bacteriorhodopsin, a light sensitive proton pump, was 
documented in 1973 (Oesterhe & Stoecken, 1973). However, the 
advent of neural optogenetics arguably started with the identifica- 
tion and expression in animal cells of channelrhodopsin-2 (ChR2) 
in 2003 (Nagel et al., 2003), followed by expression in neurons in 
2005 (Boyden et al., 2005). The key difference was that, as ChR2 



is a light sensitive cation channel, it could be used to depolarize 
nerve cells and thus generate action potentials. 

In the same period, melanopsin — a light-activated, depolarizing 
transduction pathway was demonstrated to optically activate mam- 
malian cells (Barnard et al., 2004; Melyan et al., 2005). Such 
G-protein coupled receptor approaches are very sensitive but very 
slow, with response times in multiple seconds. As such, their use is 
now being explored for non-neuronal systems (Koyanagi et al., 
2013). In 2004, the first paper demonstrating chemically assisted 
optogenetics was published (Banghart et al., 2004). In this case, a 
potassium channel was genetically modified to make it susceptible 
to binding to a photosensitive ligand with a plug. The channel 
would then remain open until light activation would extend the li- 
gand and plug the channel. Work in this field is ongoing, but is 
complicated in requiring both chemical and genetic modifications. 
In 2007, the optogenetic repertoire was expanded with the addition 
of halorhodopsin (HR) (Han & Boyden, 2007), a light sensitive 
chloride pump. This allowed for both optical stimulation and inhi- 
bition, determined by wavelength. These different forms of optoge- 
netics are summarized in the cartoon in Fig. 1. 

Since its inception, the field of optogenetics has exploded, 
being adopted by neuroscience laboratories around the world to 
study a variety of basic science questions in numerous species. 
Many review articles describing the development and application 
of optogenetics have been published (e.g., Bernstein & Boyden, 
2011; Fenno et al., 2011; Zhang et al., 2011; Mei & Zhang, 2012; 
Packer et al., 2013). In 2010, optogenetics was chosen as the 
"Method of the Year" across all fields of science and engineering 
by the interdisciplinary research journal Nature Methods (Nat. 
Meth.Editorial, 2012). 

From the perspective of visual prosthesis, one of the key advan- 
tages of optogenetic approaches is the ability to achieve targeted 
delivery to key subcircuits. Of particular pertinence to retinal pros- 
thesis is the specific targeting of ON bipolar cells (Lagali et al., 
2008). The clinically relevant method for achieving this is via a 
viral expression system (Kamimura et al., 201 1). Serotypes of lenti 
or adeno-associated viral vectors with good expression in target 
cells can be incorporated with genes to express particular combina- 
tions of opsins and promoters to improve cellular specificity. The 
most promising approaches use variants of ChR2 or HR to depo- 
larize and hyperpolarize target cells, respectively. 

Once transfected, optogenetically encoded cells require a high 
irradiance in order to be activated. The specific irradiance depends on 
expression levels, the biophysics of the opsin used and the dynamic 
range of signal required (Grossman et al., 201 1). This requirement 
could impact on tissue safety — i.e., high levels of blue light could 
cause photochemical damage (Degenaar et al., 2009). It also limits 
the range of optoelectronic stimulator options to those that can pro- 
vide sufficient light intensity (Grossman et al., 2010; Reutsky- 
Gefen et al., 2013). As will be discussed later, there are continuing 
efforts to improve opsin sensitivities with, for example, over 103 
different opsin variants available from the Deisseroth lab, a review 
of which can be seen in Zhang et al. (201 1). A review of the inverte- 
brate and small mammal implementations has been given by Kos et al. 
(2013) CatCh-ChR2, in particular, appears to reduce the light require- 
ment by over an order of magnitude (Kleinlogel et al., 201 1). 



Optogenetic photosensitization of the human eye 

Much can be learned from the last two decades of effort into devel- 
oping electronic retinal prostheses and it is worth highlighting their 
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Fig. 1. Different forms of optogenetic photosensitization in conceptual form: (A) optically sensitive ion pumps such as halorhodopsin 
(HR); (B) optically sensitive ion channels such as channelrhodopsin-2 (ChR2); (C) ion channels genetically engineered to allow attach- 
ment of optically active chemical groups; (D) optically sensitive signal transduction pathways such as melanopsin. 



successes and challenges. Two systems have effectively been 
brought to market by the 2 nd Sight and Retina Implant AG com- 
panies (Dorn et al., 2013; Stingl et al., 2013). But it has taken 
2 decades of science and engineering research to achieve this. Key 
issues include: 

• Electrical stimulus can excite but not inhibit neurons. 

• Electrodes are typically larger than individual cells, and thus it 
is challenging to target individual cells and thus specific infor- 
mation pathways. 

• As stimulation arrays get larger, the electrodes get smaller and 
new materials are required to reduce electrolytic degradation 
and thus improve device lifetimes. 

• A considerable effort has been required to create an implantable 
package with sufficient power and information transfer. 

Perhaps fundamental to all of these issues is that electronic retinal 
prostheses need to be implanted. They must thus be durable over 
many years and not cause functional loss to the surrounding tissue, 
and this significantly increases cost. As all healthcare providers 
will perform cost-benefit analyses on new products, such systems 
will need to provide some functional vision in return for their sig- 
nificant cost. It would be difficult to argue that current prostheses 
return truly functional vision. 



Optogenetic retinal prostheses are fundamentally different, as 
can be seen in Fig. 2. Clinically it would involve the injection of a 
viral vector to make specific cells in a target area of the remaining 
retina light sensitive. Then, given that the eye is transparent, the 
patient could wear an adapted virtual/augmented reality (AR) 
headset which can transmit appropriate pulses of light. No invasive 
surgery is required (though brain level visual prosthesis would 
need implantable optoelectronics). As such, the optoelectronics 
do not require the same level of biocompatibility and thus 
complex engineering as their implantable electronic counterparts. 
Optogenetic prosthesis should therefore be more straightforward to 
develop and cheaper to implement. Many of the durability prob- 
lems and cellular targeting could also be overcome. Obtaining high 
resolution should simply be a matter for increasing the resolution 
of a high brightness display and developing appropriate optics. 

As with any new technique, optogenetics is not without its own 
challenges. There are two major ethical and safety hurdles to over- 
come. There are the ethical issues surrounding human genetic 
engineering and the possible toxicity of the viral vectors used. 
Preclinical and early-stage clinical trials of gene therapies for 
various retinal disorders have largely not reported any adverse 
effects of retinal transfection (Boye et al., 2013). Two groups have 
shown long-term (over one year) expression of ChR2 in rodent 
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Fig. 2. Optogenetic visual prosthesis concept. One of the remaining layers of the retina is photosensitized, and a high radiance image is 
projected via the eye's optics. The photon peak requirements for RGCs have been taken from Bi et al. (2006), for bipolar cells from 
Lagali et al. (2008), and for cone cells from Busskamp et al. (2010). The requirement was taken as the intensity 256 (8-bit) times the 
minimum effective threshold turn on, so as to give grayscale dynamic range to normal display technology. 



retinas without serious adverse effects (Ivanova & Pan, 2009; Sugano 
et al., 201 1). Taken together, these findings are encouraging for the 
safety of optogenetic retinal prostheses, although safety data from 
larger mammals, particularly nonhuman primates, are still lacking. 
Moreover, one study in mice found (Sugano et al., 201 1) abnormal 
morphology and connectivity associated with long-term, high-level 
expression of ChR2 in the cortex (Miyashita et al., 2013). However, 
viral transfection — as would be used in clinical application — was 
associated with lower rates of abnormalities and the functional 
consequences of these abnormalities, if any, are unknown at pre- 
sent. If long-term optogene expression has similar consequences in 
the retina, it may not be an issue given the extensive remodeling 
that takes place in the degenerating retina anyway (Marc et al., 2003), 
but it emphasizes the necessity of ensuring transgene expression is 
restricted to the retina and, particularly in the case of RGC targeting 
prosthetics, that it does not spread to higher brain areas. 

Assuming that in the coming years the fundamental ethical and 
safety challenges can be overcome, there are three major technical 
challenges. The opsins need to be improved to provide better 
photosensitivity — thus reducing the light requirement (Grossman 
et al., 201 1). Vectors need to be developed to target key layers and 
cells of the retina. Finally, an efficient optoelectronic headset is 
required with a high radiance. 

The earliest attempt at an optogenetic retinal prosthesis was by 
Bi et al. (2006), wherein they transfected rdl mouse retinas with 
ChR2. Expression was retina-wide, although primarily in the RGCs, 
and they were able to elicit action potentials and visual-evoked 
potentials (VEPs) in the cortex. These results were extended to the 



Royal College of Surgeons rat by Tomita et al. (2007), who were 
also able to record VEPs as well as optomotor responses (Tomita 
et al., 2009, 2010). Since then, a number of other groups have 
achieved similar results in various models of retinal degeneration 
using a variety of different opsins, promoters, and expression tech- 
niques (Lagali et al., 2008; Lin et al., 2008; Zhang et al., 2009; 
Busskamp et al., 2010; Thyagarajan et al., 2010; Doroudchi et al., 
2011; Nirenberg & Pandarinath, 2012). These approaches can be 
separated based on a number of key decisions, two of which in 
particular are described in more detail below. 



Choice of target cell population 

The retina is a layered structure and visual information flows from 
the photoreceptors in the outermost layer, through the inner retina 
where it undergoes various processing, before finally reaching the 
ganglion cell layer, where RGCs encode information and pass it 
along to higher visual areas. In principle, it would be best to inter- 
vene as early as possible in the visual pathway, i.e., in the photo- 
receptors. RP — the key target of retinal prosthesis — causes 
inactivation and degeneration of the rod cells. This in turn causes 
degeneration of cone cells (Santos et al., 1997). However, these 
cone cells are not necessarily completely destroyed. They lose their 
outer segments and survive until very late stages of the condition. 

In 2010, Busskamp et al. (2010) demonstrated photosensitiza- 
tion of mouse cone cell inner segments by transfecting them with 
HR. HR is a better choice than ChR2 as the cones are normally 
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depolarized in the dark and hyperpolarize in response to light. 
Photon fluxes of around 10 14 photons/mm 2 s (10 /jW/mm 2 ) for 
5-10 ms were required to evoke strong responses. 

The chief drawback of targeting the photoreceptors is the low 
number of surviving cells to transfect (Stone et al., 1992; Santos et al., 
1997). In late stages of RP in particular, the surviving cones are pri- 
marily in the macula (Milam et al., 1998). As such, high resolution 
vision can be restored, but it will be restricted to tunnel vision akin to 
mid-stage RP. Additionally, in late stages of other retinal dystrophies 
there may be no surviving photoreceptors detectable via optical coher- 
ence tomography (which is likely to be an important diagnostic tool 
for determining patients' suitability for optogenetic prostheses), 
whereas the inner retina is largely intact (Jacobson et al., 2013). 

Given the relative preservation of the inner retina and in keeping 
with the idea of intervening as far upstream as possible, the next 
best cell population to target is the bipolar cells. These cells are 
important in creating the centre-surround spatial organization of 
retinal receptive fields. Thus bipolar cells come in ON and OFF 
varieties, respectively depolarizing and hyperpolarizing in response 
to light. Expressing the same opsin in both populations would 
cause conflicting responses to visual stimulation and so may not 
elicit useful percepts. Two groups have attempted to circumvent 
this problem by targeted expression of ChR2 only to ON bipolar 
cells in rdl (Lagali et al., 2008) as well as rd6 and rdlO (Doroudchi 
et al., 2011) mice, with encouraging results. In the case of the 
former, with wild type ChR2, 10 15 photons/mm 2 s (100 pW/mm 2 ) 
were required to elicit strong responses. However, with advanced 
forms of ChR2 such as CatCh (Kleinlogel et al., 2011), and per- 
haps better expression, this could be brought down significantly. 

A key issue with the inner layers of the retina is the anatomical 
remodeling and physiological dysfunction that occurs subsequent 
to photoreceptor loss in retinal degenerations (Marc et al., 2003; 
Stasheff, 2008; Stasheff et al., 201 1). In particular, extensive path- 
ological connectivity arises and strong oscillatory field potentials 
develop, accompanied by spontaneous bursting activity. It seems 
unlikely that normal retinal processing would remain intact in the 
presence of such remodeling and the signal-to-noise ratio of any 
signal passing through the retina will be diminished by the sponta- 
neous activity (Toychiev et al., 2013). Previous work suggests that 
certain features of retinal processing, such as centre-surround re- 
ceptive field organization and direction selectivity, are relatively 
preserved despite the remodeling, at least for simple stimuli (Lagali 
et al., 2008; Busskamp et al., 2010). Whether these results hold for 
more sophisticated stimuli remains to be seen. 

Alternatively, it is possible to bypass the degenerated inner retina 
and target the RGCs directly (Thyagarajan et al., 2010; Tomita et al., 
2010; Nirenberg & Pandarinath, 2012). Thyagarajan et al. (2010) 
found poor results with nonselective expression of ChR2 in RGCs, 
perhaps because their approach abolishes the distinction between 
ON and OFF pathways (Doroudchi et al., 201 1). Another possibility 
is that the visual scene has already undergone considerable process- 
ing before reaching the RGCs, so to effectively stimulate the retina 
using this approach requires replicating this processing prior to stim- 
ulation. To test this, Nirenberg and Pandarinath (2012) stimulated 
blind mouse retinas expressing ChR2 under the same promoter 
used by Thyagarajan et al. (2010) with natural images processed by 
a simple computational model of retinal encoding. They also pre- 
sented the same images, without preprocessing, to wild-type retinas 
and trained a decoder to match retinal population responses to 
images. The decoder achieved nearly identical performance when 
tested on wild-type RGC responses and on RGC responses evoked 
by their optogenetic system. 



Choice of opsin and promoter 

As discussed previously, there are a variety of optogenetic proteins 
available. One major distinction is between excitatory proteins, 
such as ChR2, and inhibitory proteins such as HR and ArchR. The 
decision whether to use an excitatory or inhibitory opsin is driven 
largely by what type of activity one wishes to evoke, for example, 
ChR2 for ON responses and HR for OFF responses (or photore- 
ceptor responses, as described above). One or multiple opsins 
could be expressed in distinct cell populations or two opsins could 
be evoked in the same cells and different wavelengths of light used 
to evoke ON, OFF, and ON-OFF responses (Zhang et al., 2009). 
This latter approach could in principle provide color opponency 
using ambient light or luminance information using a stimulator 
that takes the derivative of the incoming image and passes informa- 
tion about light increments in the blue (ChR2) channel and decre- 
ments in the red (HR) channel. 

An even more sophisticated approach targets ChR2 and HR 
separately to the soma and dendrites of RGCs (or vice versa) in 
order to recreate the centre-surround organization of healthy RGCs 
(Greenberg et al., 2011). Innovative though these approaches are, so 
far neither group has presented behavioral data to show they are supe- 
rior to simpler approaches involving a single opsin and promoter. 

Most previous work in retinal optogenetics has used classic ver- 
sions of the relevant opsins, but since their initial discovery, a mul- 
titude of new varieties with altered kinetics, mechanisms of action, 
and spectral sensitivities have been engineered (see above). ChR2 
can be modeled in terms of light and dark adaptation with lower 
performing and inactive states in the light adapted form (Nikolic 
et al., 2009). Increasing stimulation reduces those channels avail- 
able for further stimulation and it is thus difficult to achieve 
neural firing frequencies beyond 40 Hz (Grossman et al., 2011). 
Attempting to drive encoded neurons faster would simply lead to 
dropped action potentials. Faster ChR2 variants such as ChlEF 
(Lin et al., 2009) and ChETA (Gunaydin et al., 2010) allow for 
firing rates up to 200 Hz. Such improved frequency response could 
enable more precise control of retinal activity with higher temporal 
resolution, which would be particularly useful for RGC-targeting 
prostheses that attempt to mimic natural retina spiking patterns 
(Nirenberg & Pandarinath, 2012). 

Another problem with classic optogenetic proteins, especially 
ChR2, is their relative insensitivity to light. Furthermore, the 
frequency studies above showed a strong correlation between 
irradiance and frequency response. Thus, irradiance significantly 
beyond threshold is required to achieve full dynamic range. This in 
turn makes it challenging for the optoelectronics and can surpass 
phototoxicity safety limits in the eye (Degenaar et al., 2009). 

Altering the channel permeability to permit Ca 2+ ion flux as well 
as monovalent cations, as in CatCh (Kleinlogel et al., 2011), greatly 
improves the light sensitivity without sacrificing speed. However, 
detailed studies will need to be made to check for any long term 
calcium-mediated excitotoxicity to cells during prolonged stimulation. 
An alternative strategy for dealing with light insensitivity is to use 
red-shifted channelrhodopsin variants such as VChR and C1V1. 
Photochemical toxicity falls exponentially with decreasing photon en- 
ergy (i.e., longer wavelengths). As such, changing the stimulation 
wavelength from 470 nm (2.64 eV) to 590 nm (2.1 1 eV) changes the 
safe irradiance limit by two orders of magnitude (Degenaar et al., 
2009). Early red-shifted variants (Mattis et al., 2012) were not as 
sensitive as some advanced blue variants, but both the sensitivity 
and response kinetics have improved with more recent red-shifted 
variants such as ReaChR (Lin et al., 2013). 
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In summary, the variety of optogenetic tools available allows 
considerable scope for improving existing optogenetic retinal pros- 
thetic approaches, but each comes with its own caveats and trade- 
offs. It could be that the challenges in developing fast and efficient 
opsins may also face challenges with immune rejection. As such, 
melanopsin — a light sensitive signaling transduction cascade could 
offer an alternative. As most prior work in this area has used wild- 
type variants, it remains to be seen whether and how novel opsins 
can lead to improved retinal prosthetic strategies. Some demonstra- 
tions on animal models have been achieved (Lin et al., 2008), but 
they would need significant fundamental alteration to speed up 
their kinetics from multiple seconds to milliseconds. Nevertheless, 
as their TRP channels already exist on the membrane of cells in the 
human retina, a long-term immune response would not be expected 
from their use. 



The optoelectronic headset 

In the late 1960s, virtual reality systems were created for the 
American airforce. These consisted of headsets which projected a 
virtual world into the user's eyes. In the 1990s, consumer versions 
were made, though with little commercial success. Now, with 
developments in various mobile technologies, there has been a 
resurgence in the field with major companies such as Google 
creating systems. The particular interest is in AR — projecting 
synthetic information over the real visual scene. Such AR is actually 
very similar to what is required for optogenetic retinal prosthesis. 
There needs to be an image acquisition system (camera), some 
form of image and retinal processing, a high intensity display, and 
an optical projection system to the retina. It all needs to fit into a 
wearable headset and operate under battery power for the course of 
a day. Much can be taken from existing commercial systems, but 
the display radiance and processing requirements are significantly 
different from existing AR systems. 

Light stimulation systems 

The in vivo studies in recent years described above give a require- 
ment of 10 12 /mm 2 photons or 0.4 /jj/mm 2 over a period of 10 ms. 
This is less than the original in vitro requirement of 10 /J/nim 2 but 



still challenging for current display systems. When optical efficiencies 
are taken into account, display brightness' of 100-500,000 cd/m 2 
are required, when typical liquid crystal (LCD) or organic light 
emitting diode displays emit around 1000 cd/m 2 (Degenaar et al., 
2009). Realizing what is in effect a high brightness display can be 
achieved by either creating a singular intense beam which is then 
spatially patterned, raster scanning beam of light across the target, 
or by creating an array of microemitters. 

Spatial light modulation is common in consumer projection 
systems for presentations. Typically, many will have a high bright- 
ness tungsten lamp with light shining through a liquid crystal 
screen (LCD), or reflected off either micromirror array (DLP — digital 
light processing) or reflective liquid crystal array (LCoS — liquid 
crystal on silicon). The light is then imaged onto the final target. 
Bright regions are simply those that have allowed all the light 
through, whereas dark regions have had their pixels attenuated. 
Some researchers have also adapted such displays for in vitro 
optical neural stimulation. As tungsten lamps are very inefficient, 
and most light is wasted by the attenuation system, such systems 
are not directly scalable to battery-powered retinal prosthesis head- 
sets. Compact laser or high luminance light emitting diode (LED) 
systems can significantly reduce the energy cost and can be com- 
bined into compact packages for head-mounted displays. But there 
is still an issue with light wastage and thus battery life. 

Laser scanning is akin to the old style CRT televisions where an 
electron beam was scanned across a phosphor surface to create an 
image. In the case of a laser, mechanical or electro-optical modula- 
tors can be used to scan the beam and deflect it off the image for 
dark areas. The efficiency is thereby the same as for LED/laser-DLP 
systems. An alternative approach involves computer generated 
holography that recently has been successfully used for in vitro 
patterned neuronal stimulation (Reutsky-Gefen et al., 2013). 
Holography should in theory be considerably more efficient than 
DLP projection, as it should utilize most of the light, although the 
processing requirement is increased in order to perform real-time 
hologram computation. 

The alternative to laser projection is to utilize high power LED 
arrays as shown in Fig. 3. Light in LEDs is produced when positive 
and negative charge carriers combine to form a photon. Increasing 
the charge carrier concentration can therefore increase the light 
output. However, there are limits for given architectures and 





Fig. 3. High brightness gallium nitride micro-LED arrays for retinal prosthesis. (A) Modules which are scalable to an existing head- 
mounted display system; (B) the electronic module; (C) an illuminated array on in vitro culture. 
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semiconductor materials. As such, gallium nitride is presently the 
platform of choice for much of the present white and blue high 
brightness LED light sources. For the purpose of retinal prosthesis, 
these high brightness LEDs can be patterned into arrays of uLEDs 
to form a display. Such systems have been demonstrated in papers 
by our team (Grossman et al., 2010), incorporating CMOS con- 
trollers, gallium nitride uLEDs (McGovern et al., 2010), and 
microlenses (Chaudet et al., 2013), and can be seen in Fig. 3. The 
advantage of such systems is that they are directly emissive, only 
producing the required light and each pixel directly correlating to 
required stimulus point. The challenge is that such technology is 
not as mature as for DLP systems and still needs development. 



Processing requirements 

The holy grail of optogenetic retinal prosthesis research would 
be a particular combination of opsins and promoters that, when 
transfected, restores normal or near-normal vision using only 
ambient light. However, between cell death, inner retinal remodeling, 
imperfect transgenic expression, reaching this goal at least in the 
initial iterations of retinal prosthesis is unlikely. Hence, there will 
be a need to modify the visual information to maximize the 
useful information content transmitted to the patient. Furthermore, 
depending on the target cell population, some form of retinal 
processing will be required in addition to standard display encoding 
and pulse modulation. This is summarized in Fig. 4. 

The first stage is the same for any AR system. A camera acqui- 
sition system needs to send visual information to the display at 
high speed without significant lag. Furthermore, automatic gain 
control, contrast and basic image enhancement techniques that 
exist in most mobile camera systems need to be performed. One of 
the problems that plagued early virtual reality was induced motion 
sickness (LaViola, 2000; Nichols and Patel, 2002). Initially, it was 
thought that it correlated to significant time lags between the head 
position and the display position, coupled with motion blur. More 
recent studies however indicate that users can adapt to time lags of 
up to 200 ms (Moss et al., 201 1), though this would be undesirable 



from an aesthetic perspective. To reduce motion blur, the com- 
monly accepted video rate is 25 frames per second. High end dis- 
plays are considerably faster, up to hundreds of Hertz, but acquiring 
data from imaging chips at such high speeds reduce their signal-to- 
noise ratio. This can affect performance under poorer lighting. 

The second stage is to enhance the useful information provided 
to the user. If the target cell group to be photosensitized is the 
degenerated cone cells, then the returned sight will be a tunnel 
vision. If bipolar cells or RGCs are targeted, then (without a fovea) 
contrast sensitivity and resolution may be poor. The returned sight 
could therefore be akin to that of someone with mid-stage RP or 
AMD, respectively. For tunnel vision, dynamic scene compression 
techniques could be used to nonlinearly spatially compress the 
visual scene such that non-important features are compressed 
leaving important features the same size (Al-Atabany et al., 2010Z?; 
Al-Atabany et al., 2013). For poor visual acuity and contrast sensi- 
tivity, cartoonization of the visual scene whereby non-important 
textures are compressed could improve the contrast and thus the 
clarity of the transmitted scene (Al-Atabany et al., 2010a; Al-Atabany 
et al., 2013). 

The third stage is to take into account any retinal processing 
which is bypassed by the prosthesis and the effects of any remodeling. 
The retina is not simply a camera. It is the first stage in a system to 
understand the world and as such extracts particular stimulus features 
(Gollisch & Meister, 2010), which are sent to the visual cortex for 
higher level analysis. As a computational system it is particularly 
tractable for analysis and modeling (Meister & Berry Ii, 1999). 
Consequently, there is a vast literature on computational models of 
retinal processing and encoding. These range from the extremely 
simple, such as the classic difference-of-Gaussians (DoG) model 
of the RGC receptive field (Rodieck, 1965) to highly detailed 
compartmental models of specific retinal circuits (for example, 
see Escobar et al., 2013 for a review of such models pertaining 
specifically to direction selectivity). Lying somewhere in between 
these is the linear-nonlinear (LN) model employed by Nirenberg 
and Pandarinath (2012), which comprises a linear filter kernel 
followed by a static nonlinearity (Chichilnisky, 2001; Paninski, 
2004; Paninski et al., 2007) and is considered in some circles as a 




Fig. 4. Visual prosthesis coding stream. (A) The visual encoding requirement depending on which cell type is being stimulated (adapted 
from Al-Atabany et al. (2013)); (B) in the future it may also be possible to explore imaging of additional wavelengths and integration of 
AR concepts (taken from Song of the Machine (Jain et al., 201 1)). 
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'standard model' of retinal computation (Carandini et al., 2005). 
Such models become increasingly important with increasing reso- 
lution as transferred images move from simple lines and shapes to 
complex features. 

The key issue with all of these models is the trade-off between 
quality and accuracy and computational complexity. Too complex, 
and the required processing could no longer be performed in real 
time by portable processing units. Models like the DoG can be 
implemented using computationally cheap convolution methods 
and are often employed by the retinomorphic engineering commu- 
nity (e.g., Mead & Mahowald, 1988; Banks et al., 2005; Martinez 
et al., 2009). Models like the LN cascade (Nirenberg & Pandarinath, 
2012) could be calculated efficiently if tight constraints are placed 
on the processing boundaries. Otherwise the complexity scales 
linearly with the number of neurons simulated. 

Finally, a pulse encoding scheme is required to emulate signal 
intensity. In many display systems, it is problematic to implement 
analog range controls per pixel. Mismatch between transistor prop- 
erties would inevitably lead to large variations in intensity. Also in 
some systems (e.g., DLP), the light beam either points to its target 
or is deflected away. Thus, intensity modulation is via pulses of 
light of a fixed intensity. The longer the pulse, the greater the inten- 
sity. Such pulse width modulation could be combined with the 
biophysics of the channelrhodopsin, i.e., if dark adapted a shorter 
pulse may be sufficient and vice versa for light adapted (Grossman 
etal., 2011). 

Since the late 1980s the neuromorphic community has suggested 
silicon chips with fixed retinal architectures as a solution to achieve 
high speed processing at ultra low power (Mead & Mahowald, 
1988; Banks et al., 2005). These may even be combined with the 
optoelectronic stimulator in a single pipelined chip (Huang et al., 
2009). In more recent years, the power of mobile graphics proces- 
sors is increasingly allowing the powerful computation required 
for retinal prosthesis, with battery life measured in hours. Such 
systems have the added benefit of being programmable and thus 
being adjustable and responsive to the individual's preference. 
They will continuously improve in the coming years with the 
increasing interest in AR. 



Future perspectives 

In the coming few years, we will see the first human optogenetic 
trials. It is no coincidence that this will be performed for retinal pros- 
thesis prior to other conditions. The human eye is transparent, 
removing requirements for implantation. In addition, should there be 
an immune response, the removal of the eye of a blind individual will 
not cause any further functional loss. Nevertheless, there are serious 
challenges. RGCs project to the brain, so in the first instance, these are 
unlikely to be targeted, lest there is an immune response in the brain 
itself. Should there be such a response, it might considerably delay 
progress in the field until non-immunogenic alternatives are found. 
Despite these unknowns, optogenetic approaches to retinal prosthesis 
hold the potential to bring back significantly improved vision — in 
particular foveal vision (not possible with electronic approaches). It 
can also be implemented at significantly lower cost compared to 
electronic approaches. If we consider the cost of intravitreal injections 
for macular degeneration, and the cost of AR headwear, then perhaps 
such systems could be at least an order of magnitude less expensive 
than their implantable counterparts. 

Finally, as systems become more advanced and commonplace, but 
not yet returning full vision, there will be scope for user interface 



development. Visual enhancement techniques, utilizing potentially 
even infrared and UV wavelengths beyond normal vision, could pro- 
vide benefit. The effect of popular culture interestingly is also play- 
ing a part. The success of the 2009 Avatar film revitalized interest 
in 3D head-mounted displays and the success of smartphone tech- 
nology did the same for AR. Advances in both technologies, largely 
driven by consumer interests rather than academic research, will in 
turn spur improvements in optogenetic prosthesis headset design. 
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